lADIS International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2013) 


ISSUES OF LEARNING GAMES: 
FROM VIRTUAL TO REAL 


Thibault Carron\ Philippe Pernelle^ and Stephane Talbot^ 

‘Lip6 Lab Paris 6 & Universite de Savoie 
^Disp Lab, Universite de Lyon 
^Universite de Savoie 


ABSTRACT 

Our research work deals with the development of new learning environments, and we are particularly interested in 
studying the different aspects linked to users’ collaboration in these environments. We believe that Game-based Learning 
can significantly enhance learning. That is why we have developed learning environments grounded on graphical 
representations of a course. These environments allow us to set up experiments with students in our university. The 
emergence of online multiplayer games led us to apply the metaphor of exploring a virtual 3D world, where each student 
embarks on a quest in order to collect knowledge related to a learning activity. In the environment, each part of the world 
represents a place, sometimes a collaborative place, where students are supposed to acquire a particular concept. Learning 
objects, artefacts or collaborative tools may be present in each location and a correct answer to a specific exercise gives a 
key to the students, allowing them to access other activities. 

Although the students appreciate this approach, there is a lack of assessment of know-how-type skills, especially for the 
teacher. Indeed, certain domains present the particularity of exhibiting both theoretical knowledge and practical know- 
how (operations in manufacturing or medicine, for example). For such contexts, the current Learning Games are not 
efficient concerning this second point: a unique and full digitalisation of these objects alone is not sufficient to guarantee 
both effective learning and assessment of the techniques. We consider this as a gamification problem. As a matter of fact, 
in industrial domains, the learning processes are often based on the use of certain objects that are difficult to include in a 
learning game. Moreover, although some games are collaborative, an effective collaborative activity is more effective in 
a real context. Our objective is then to facilitate the transfer by integrating a new object present both in the game and in 
the classroom. 

In this paper, we focus on the way to integrate the use of real objects into the learning game environments. In our 
approach, we develop a digital copy of the object that will help to exhibit the specific know-how that must be evaluated. 
It is then easier during the gamification process to include in the learning scenario certain parts that are relevant to the use 
of such an object and that are achieved in a real context. First, we describe briefly the “Learning Adventure” 
environment: a generic game-based platform. Then, we explain what the new issues of such contexts are, and how we are 
able to setup a collaborative learning session with both virtual and real objects. The third part concerns the application to 
a concrete problem: to identify and manage Nonconformities thanks to a Product Lifecycle Management tool. A real 
experiment has thus been carried out at our university in the PLM domain with the help of a multi-touch tabletop, to 
validate the feasibility of the approach and illustrate our point. 
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1. INTRODUCTION 

Nowadays, compared to traditional teaching methods, Learning Management Systems (LMS) offer 
functionalities that are recognized as being valuable from different points of view. For instance, students can 
learn at their own speed. These environments also allow the teacher to evaluate specific activities in a 
uniform way. However, although they enable powerful features, they also receive major kinds of criticism 
(lack of awareness, few collaborative or regulation possibilities (Kian-Sam 2002)). Moreover, some students 
tend to consider LMS as unexciting (Prensky, 2000). 
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That is the reason why, observing the emergence and success of online multiplayer games with our 
students, we decided to adopt a learning game approach, by developing our own game environment and by 
using it as a support for some learning sessions. Agreeing with Vygotsky’s school of thought and activity 
theory (Vygotsky, 1934), we consider that the social dimension is crucial for the cognitive processes 
involved in the learning activity. Indeed, we think that the way of acquiring knowledge during a learning 
session is similar to following an adventure in a Role-Playing Game (RPG). The combination of the two 
styles is called MMORPG (Massively Multiplayer Online RPG) and offers a good potential for learning 
(Galameau, 2007) reformulated as MMOLE (Massively Multiplayer Online Learning Environment). 
Nevertheless, although the students appreciate this approach and that Game-based pedagogic tools can 
significantly enhance learning, there is an obvious need for realistic and reliable assessment about students’ 
skills, actions or behaviours, especially for the teacher. Indeed, for certain specific domains, the teacher needs 
to evaluate his/her pedagogical session according to two specific points that are particularly difficult to assess 
via such a learning environment. Certain domains present the particularity of exhibiting both theoretical 
knowledge and practical know-how (operations in manufacturing or medicine, for example). For such 
contexts, the current Learning Games are not efficient concerning this second point: a unique and full 
digitalisation of the objects alone is not sufficient to guarantee both effective learning and assessment of the 
techniques (Schrier, 2006). 

In this article, we will focus on these two points: the know-how and the collaborative behaviour. We will 
first describe the Game-based learning environment that we have developed, and next focus on integrating a 
new object concerning the two sides of the world - real and virtual - in order to set up a mixed reality learning 
game. The main point of this article is how to take into account the new issues due to the mixed reality, and 
in a secondary point how to gather in the same environment information concerning general learning 
concepts and business concepts. Finally, in the last part, we will try to illustrate most of these points by 
means of a concrete example and set out a way via a real experiment to identify and manage non- 
conformities thanks to Product Lifecycle Management in such a learning environment. 


2. LEARNING ADVENTURE ENVIRONMENT 

Learning Adventure is a Game-Based Learning Management System representing a 3D environment where 
the learning session takes place (see figure 1). A particular map (buildings, enterprise areas but also the 
environment with lakes, mountains and hills) is dedicated to a particular learning activity, for a particular 
subject. Each part of the map represents the place where a given (sub-) activity can be performed. 



Figure 1 . An Industrial Map in the Learning Adventure Environment 

The map topology represents the overall scenario of the learning session, i.e. the sequencing between 
activities. There are as many regions as actual activities, and the regions are linked together through paths 
and NPC (Non Playable Character) guards, showing the attainability of an activity from other ones. 
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An example of a scenario seen as a map topology is presented in figure 2. Similar models that link 
pedagogical issues with game elements can be found with a more general point of view in (Amory 1999) and 
more precisely concerning this approach in (Carron 2008). 



Eigure 2. An example of a Scenario Seen as a Map Topology 

As explained previously. Learning Adventure is based on a role-play approach (Baptista 2008). Players 
(students or teachers), possibly represented by their own avatars, can move through the environment, 
performing a sequence of sub-activities in order to acquire knowledge or/and solve problems (here finding 
and identifying non conformities, see figure 2). Activities can be carried out in a personal or collaborative 
way (see (Dillenbourg 1996) for a list of cooperation abilities); one can access knowledge through objects 
available in the world, via help from the teachers, or from work with other students. 

Although such game environments and characteristics are well known to our students, the so-called 
«digital natives», some reminders are always proposed at the beginning of the pedagogical session. This first 
playable part called the Newbie Park allows us to describe the main functionalities, and explain the use of 
certain specific collaborative tools that are present in this game or this particular learning session. Moreover, 
as in many collaborative sessions, this part can be seen as a “warm-up activity” in order to get students’ 
minds into an appropriate “ready to play for learning” state. As regards Learning Games, our experience 
shows that such a first step is crucial, as is a final debriefing phase in order to help cement the knowledge 
acquired (Garris et ak, 2002). 

The environment is generic in the sense that it is not dedicated to a particular teaching domain. With the 
help of a pedagogical engineer, the teacher adapts the environment before the session by setting pre- 
requisites between sub-activities and by providing various resources (documents, videos, quizzes) linked to 
the course. Experiments have been set up for learning English as well as Project Management or Object 
Oriented Concepts in Computer Science. The collaboration takes place in L.A. by constituting groups of 
users. The NPC will give objectives to the members of a group and allow them access to collaborative tools 
such as white boards, file boxes or a “collaborative plan elaborator” similar to a structured discussion forum. 
It is then possible to construct group knowledge with specific tools. Naturally, in order to communicate with 
other players a chat tool is available (bottom left comer in figure 1). 

It is possible for the students to embark on several knowledge quests in parallel. That is why it is 
important for them to be aware of the quests currently active. In order to facilitate navigation inside the game, 
the objects or NPCs to be reached in the active quests are displayed on a compass (upper right corner in 
figure 1). Moreover, one can easily reach one’s user model and consult the current content of one’s bag (see 
bag window open and icons on the right hand part of figure 1). Nevertheless, as explained previously, certain 
specific skills (particular manipulation, know-how) or behaviours (collaboration) are difficult to assess by 
staying only in the virtual world, and underscore the necessity of reintegrating some parts of the game into 
the real world. Prom our point of view, the relevant work is not trivial in nature and raises some new issues 
that have to be taken into account in order to keep the advantages of serious games (immersion, motivation). 
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3. ISSUES OF THE GAMIFICATION PROCESS WITH MIXED REALITY 
GAMES 

In order to illustrate our point, we will take the example that will be the focus of the experiment described 
below. 

In quality management, a nonconformity is a deviation from a specification, a standard, or an 
expectation. Certain specific information systems known as PLM (Product Lifecycle Management) are 
dedicated to providing information about such defects. It is quite difficult to identify, via a serious game, 
such skills that generally impose debates between the students/players featuring descriptions of the problem. 

For example, our proposition was to create both a specific tool and an object able to exhibit such skills. 
For our example, it was important for a student to be able to present to the other ones and observe with them 
the possibly defective objects: we chose to create a multi-touch tabletop in order to provide a collaborative 
tool for that purpose. Nevertheless, depending on the context, many other interface objects could be 
envisaged (Feedback/Data gloves (haptic devices), 3D goggles, Kinect, physical objects (“tangible things”), 
QR Code, NFC Tag, etc.). 



Figure 3. Students Debating About Nonconformity around the Table and the Representation of the Corresponding Scene 

in the Learning Game 


The problem of the gamification process (i.e. the transformation of common business processes into one 
or several game scenarios) is already well identified, but concerning MRLG (Mixed Reality Learning 
Games), some brand new problems have appeared. 

First of all, to remain coherent, it is important to find a way to pass “naturally” from the virtual world to 
the real one. Our idea was to use a specific object that existed in both worlds. This object would be used to 
support this transition smoothly (like a transitional object in the psychological domain). Another good side 
effect of this object is to help the user to reinvest their acquired knowledge more easily outside the virtual 
game. 

A second problem concerns the access to this object: as we are working in a collaborative session, several 
students may require access to this object at the same time. When this object represents a (group) exclusive 
(or non-shareable) resource, the learning scenario has to be adapted to such a configuration in order to avoid 
any delay. As a matter of fact, any delay would result in instant disengagement of the students. 

Moreover, it is now well known that good collaboration is effectively achieved with a group of 3-5 
persons at most. We have then to ensure that the players of the same groups will progress together at the 
same speed and will be synchronized thanks to the scenario evolution. 

A Serious Game concerning change management has already been proposed as a solution for specific 
training in matters of industrial problems. The purpose here is to be more general and go further concerning 
the gamification process by including mixed reality in such learning games. Mixed reality has allowed us to 
take into account new needs, but it also means that some new problems have to be addressed. 

As seen in table 1 , for each problem relative to mixed reality that we have detected, we have proposed a 
solution. 
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Table 1. Examples of Mixed Reality Issues 


Domain 

issues 

Virtual World 

Real World 

Pedagogy 

Concrete assessment 

(know-how, manipulation, 
gestures) with a specific device. 

Virtual representation of 
the specific device 

(no evatuation) 

Real Object/ Specific 
Device 

(effective work) 

Entertainment 

Keep the learner immersion 

Coherent Story leading to 
the specific object 

Integration of clues 
and challenges that 
remind us that we are 
in a game. 

Collaboration 

(intra-group) 

Synchronize students in the 
same groups 

Propose some optional 
mini-quests 

Small groups with an 
active role for each 
one. 

Collaboration 

(extra-group) 

Synchronize groups of students 
(no wait for access to a group 
exclusive resource) 

Change the order of the 
quests. 

Access to the real 
world 

Content 

Assessment of content 

Indicators thanks to traces 

Camera or results 
reintegrated into the 
game via digital input. 

Scenarization 

Retain a complete game 
feeling. 

Beginning and end in the 
game. 

Small moments in the 
real world. 


In fact, the main points in the table above are not the solutions that are proposed for illustration but rather 
the issues. These solutions are not exhaustive, and must be adapted to perfectly fit with the context. 
Nevertheless, these issues have to be resolved because setting up such experiments is very costly and 
generally cannot be easily done again. Indeed, logistical problems increase with collaboration and mixed 
reality needs. It is important to anticipate these problems and keep motivation and immersion at a high level. 

These considerations have permitted us to prepare a serious game in mixed reality that has been 
experimented in a real context. The following section describes the content of this scenario. 


4. SCENARIZATION AND APPLICATION TO NON-CONFORMITY VIA 
PRODUCT LIFECYCLE MANAGEMENT 

The implementation of a PLM system significantly alters the organization of the company, particularly in the 
context of SMEs (Small and medium-sized enterprises). Resistance to change (individual and collective) 
appears naturally during the start-up of this type of system (Kadiri et al. 2009). Our learning environment is 
defined for supporting change management in industrial enterprises and to help companies to effectively 
support their staff in adapting to the changes brought about by redesigning their information systems. The 
challenge is now to train the employees for them to have the ability to detect and manage nonconformities 
with such an attractive environment. The relevance of this serious game is twofold; the use of video game 
techniques to teach the use of PLM to detect nonconformities and evaluate the contribution of mixed reality 
in assessing particular skills concerning the identification of nonconformities. 

Concerning the assessment, all tools are equipped with specific probes in order to obtain information 
about the learning progression of the students (Marty et al., 2012), even for the specific device where the 
results are reinvested in the game. For example, all the actions carried out through the interface are collected 
in the game. We are thus able to know who has made an incorrect manipulation of the business tool or the 
business process, provided a positive or a negative comment, asked for an explanation, etc. For very 
particular assessment (that is not numerically observable), we also use camera and real observers present in 
the room. Generally, the evaluation of the use and ownership of content by a user is needed. 

For this collaborative scenario (see figure 2), we have defined two important activities (Al and A2) that 
are the main quest in sequence and three sub-activities (A 1.1, A 2.1, A 2.2) that are optional (for 
synchronization purposes: see table 1). We can add AO for a starting activity and A4 for an ending one. 

The scenario seems basic, but the sequence was in reality not fixed, and three ways of achieving it were 
expected. The narration proposed by the NPC changed with each group in order to enable access to the multi- 
touch tabletop at different moments of the scenario and avoid any delay. 

One of the difficulties of such games is the definition of industrial scenarios. The aim of the modeled 
scenarios is to understand the use of PLM in identifying and managing nonconformities. The table below 
shows the steps concerning the main features that we want to evaluate. 
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Table 2. Steps of Usage Scenarios 


Steps of the scenario 

Example 

Observe the environment 

A battery silo is different from the others 

Identify five possible NCs 

This saber is too short 

Understand the use of PLM as a tool for 
collaboration 

Have a look at the specific orders that could 
explain something unusual 

Collaborative Exchange 

Identify formally three NCs 

Validation 

Input/submit the proposition to the chief 


This process is carried out in the Serious Game with a connected PLM system as well as access to a 
collaborative platform in order to reach some additional learning resources (videos, tutorials). The actions 
that can be performed in the game are the following: discussion with colleagues and watching a training 
presentation, collecting information (taking pictures) in order to debate about nonconformities and visualize 
these pictures on the multi-touch tabletop, filling in the nonconformity form, retrieval of information from 
the archives, etc. The results are sent via the PLM tool to the teacher who validates or invalidates the choices. 

This collaborative scenario has been experimented several times in an ecological context, as we will 
describe in the following section. 


5. EXPERIMENT DESCRIPTION AND RESULTS 

The experiment in real conditions concerning Nonconformities and Product Lifecycle Management has been 
realised in order to validate our approach at the Lyon Institute of Technology with master’s students. 

5.1 Conditions and Methodology of the Experiment 

This experiment was carried out in 2013 at the University of Lyon with co-located settings. During the 
experiment, a group of 12 students (three groups of four persons simultaneously) with their teacher were 
present successively in a classroom equipped with 12 computers. Concerning the social presence perception 
(see (de Kort, 2008)), players were oriented away from each other, limiting mutual eye contact, natural 
reciprocation of approach or avoidance cues and mirroring. The students were between 24 and 50 years old 
(PLM Master’s) and for the most part familiar with computer use. Each student accessed the virtual 
environment through his/her workstation, and had a personal (adapted) view on the world. These students 
used the environment for approximately two hours. They were explicitly allowed to communicate through 
the chat tool provided with the system and were warned that they would be observed concerning the use of 
the system. At least two observers were present in the classroom. The students were free to refuse this 
observation (the same practical work was available outside the learning environment), but everyone agreed to 
follow the proposed protocol. Finally, the first part of the session consists of a fifteen-minute introductory 
section called the «newbie park», used in order to present and explain step by step the basic functionalities of 
the game (collect bag, quest book, skill book). 

The same experiment has already been carried out four times (48 students) with similar results that we 
will explain in paragraph 5.3. 

5.2 Support for Pedagogical Objectives 

The pedagogical objectives explained in the preceding section are supported by a story that guides the 
knowledge quest with the help of metaphors. For motivation purposes, we integrate the scenario into a 
futuristic world. Indeed, the challenge is encouraged through NPCs which propose a coherent contest. 
Immersion is reinforced when the users' actions have a direct impact on the objects of the world. Finally, the 
teacher was present in the game via an avatar: it was possible to chat with him, to ask for help for example. 
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5.3 Evaluation and Results 

Concerning this experiment, two means of evaluation were chosen; 

• Quantitative, thanks to collaborative indicators elaborated with traces left by the users when 

collaborating, 

• Qualitative, with live questions at the end of the session and explicit feedback from the 

teacher/PLM expert. 

At the end of the experiment, the students were asked to answer questions to give feedback about their 
feelings concerning their work session. The questions (ranking and open-ended ones) evaluated aspects 
relating to several parts of the learning game (pedagogical content, business concepts, affordance of the 
objects or tools, scenario-story, immersion, collaborative activities and specifically collaborative tools, and 
user model evolution). The final question let the students propose improvements concerning weak and strong 
points of the game. For example, they found that other sub-quests should be proposed, and some people were 
very interested in exploring the map (and found jokes or amusing allusions). This classification of gamers is 
well-known (Yee, 2007). 

The initial objective concerning nonconformity identification was achieved. The students were 
unanimous in preferring to work with this environment rather than doing conventional practical work on 
workstations, and more generally were very enthusiastic about this kind of experiment. 

From the teacher’ s point of view and in the light of previous experiments, it was naturally mandatory to 
have specific tools supporting him in the monitoring task with the help of an updated user model for each 
student. As a matter of fact, the teacher is present in the game to validate propositions of nonconformity, but 
also to give access (/or deny access) to the right students to the “debate room with the real tabletop”. 

The results were satisfying but the setting-up of such tools and experiments is, as always, extremely time- 
consuming for the teacher - indeed, even more so than usual. We will see later whether all these facilities 
may be reused easily in other domains. Other experiments with the same environment but applied to a 
different domain (electronic teaching with arduino electronic shields) are currently planned with a larger 
public (60 students) and in another institute of technology. 


6. CONCLUSION 

In this article, we have illustrated a way of integrating real tools into a collaborative session of a learning 
game in order to be able to assess particular skills or behaviors. Thanks to information found both in the 
game during the pedagogical session and through the use of a professional business tool, and also with the 
use of a multi-touch tabletop, we validated this approach with an experiment with our students concerning 
the management of nonconformity with Product Lifecycle Management Systems, the said experiment being 
carried out in the Learning Adventure environment. This environment is collaborative, multiplayer and fully 
observable thanks to traces left during the game. These traces allow us to elaborate collaborative indicators. 
Moreover, through the feedback collected from these experiments, we are able to obtain new factual 
indicators of collaboration by exploiting the traces left by the users. 

As stated previously, this environment is still being developed and will be aimed at proposing both 
specific collaborative tools and facilities to help the teacher to regulate a learning session. Moreover in 
future, we would like to see how teamwork may be self-regulated thanks to specific functionalities of 
collaborative tools. We have shown an example based on several players interacting and communicating in 
the same environment, but with the use of real teams and roles, we may also imagine self-regulation, team- 
regulation or auto regulation by the use of specific rules, as can be the case in Artificial Intelligence. Self- 
regulation, co-regulation and socially-shared regulation are precisely described in (Hadwin 2010). 

Naturally, certain drawbacks persist: we must recognize that it is very difficult for the teacher to be 
present in the game, help the students and regulate the session, even with these specific tools. For the 
moment, the indicators that are present are not very well-integrated, because they are very time-consuming to 
develop. We currently think that we can develop some specific generic indicators dedicated only to classical 
fields of a domain. 

An interesting perspective could be to develop and propose directly within the indicators some basic 
regulation actions such as “play specific PLM video”, “propose new (adapted) content or new (adapted) 
scenario” or “enable/disable such facility/ies for this student”. 
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Finally, in another project called SLI (Serious Lab for Innovation), again supported by the French 
Ministry for the Economy, Industry and Employment, we also applied this work to another domain: 
Innovation in industry. Naturally, another scenario was imagined, developed and proposed for that purpose. 
Some qualitative results concerning concepts and methodology about innovation are still under processing. 

Erom a more general point of view, “conducting change” is a key issue both for industrialists and 
students, who will experience such a challenge later on. A generic serious games approach will help us to 
link the gap between education courses (a theoretical approach thanks to pedagogical resources left in the 
game) and concrete experimentation in a professional context thanks to the integration of a real business tool 
(access to interfaces via web services). 
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